Autonomic Computing: Learning to Repair Systems Effectively

This research aims to integrate temporal difference learning methods into an autonomic learning system. The current Rainbow architecture models an adaptation engine that examines the system state to assess problems and determines a best course of action based on pre-programmed expertise. However, when expert knowledge is not readily available for a system, learning proper actions without a priori knowledge would often be more useful in practical situations. This report characterizes initial testing of a new learning engine designed to learn the proper corrective actions to take in order to repair problems in systems. Specifically, reinforcement learning methods can enable a system to learn the proper actions by actually trying out actions and seeing how well they do. This project compares Q-learning and SARSA temporal difference learning algorithms with the existing expert algorithm on the Carnegie-Mellon designed Rainbow simulated system. Since they approach reinforcement learning in a different way, comparing the success rate and speed of the algorithms on various scenarios will provide evidence about their efficacy. My research aims to clarify the benefits and costs associated with these algorithms.